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Introduction 

During early accreditation visits by the Australian Medical Council (AMC), staff in the School of 
Medicine (SoM) were asked to demonstrate how and when AMC student outcome statements were 
being integrated into the MBBS course. Although staff were confident that this was occurring, 
they were at a loss to provide tangible evidence. As a result, the School Executive committed to 
developing a curriculum mapping system (CMS) that could systematically demonstrate alignment 
between course outcomes, the AMC outcomes as well as other curricula elements. The question 
was, however, where to begin? 

A thorough scan of the literature was conducted to see whether commercial products existed that 
might provide a ready-made curriculum mapping solution for the School. Although a few were 
identified, Rubicon Atlas (by Rubicon International), Illios (open source software by the 
University of California, San Francisco), and eMed (by the University of New South Wales) were 
explored in-depth, primarily based on the willingness of their product owners to engage in 
conversation and share their experiences. However, despite the potential offered by these systems 
it was apparent that the time it would take to customise them to ‘fit’ the SoM’s needs as well as 
the issue of on-going support and maintenance suggested that a complete build of a custom-made 
solution was necessary. 

Early attempts at a CMS saw staff entering ‘curriculum data’ (ranging from learning objectives to 
teaching guides to timetabling information) into Microsoft Access but it was discovered fairly 
quickly that more ‘logic’ needed to be imposed onto the curriculum for this or any other 
application to work effectively. Also, the overlap between what the CMS was aiming to do, and 
what the University’s learning management system (Blackboard) was already providing led to 
confusion about the ultimate purpose of curriculum mapping; and what the expected outcome 
should look like. The eventual failure of Microsoft Access as a solution for mapping the 
curriculum was more to do with a poor grasp of the actual problem, rather than the application 
itself. 

These early attempts at creating a CMS were deemed unproductive, wasteful and were frustrating 
for all involved. However, upon further reflection, although little tangible progress was made, each 
failed attempt led to the articulation of questions that were critical to shaping Prudentia and, more 
importantly, to the development of the School’s curriculum framework. In an effort to better 
understand the purpose of the curriculum map and what value it might offer staff and students, the 
following questions emerged: 

• What is curriculum mapping, and why is it necessary? 

• What is the logic underpinning the SoM’s curriculum? 

• What curriculum data should be mapped? 

• How does a curriculum mapping system differ to a learning management system and 
should these two systems interface? 

In attempting to answer these questions, it became abundantly clear that whilst the platform for 
storing and communicating the curriculum and its relationships should be user-friendly, the 
technology itself should not dictate the structure of the curriculum. Rather, the technology should 
reflect and support the School’s curriculum framework and processes. Reeves, McKenney & 
Herrington (2011) noted that when employing technological solutions to educational problems, the 
technology can become the primary focus, often to the detriment of the quality of the outcome. As 
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such, an agile approach to software development was adopted whereby an application evolves in 
an iterative process of planning, building and testing against requirements. This adaptive approach 
was essential, given that there was no exemplar that the SoM could use to guide development. 
Instead, development was guided by gradually finding answers to the above questions. 

The development team comprised the Associate Dean Teaching and Learning (project owner), two 
software engineers, two academic staff (representing the preclinical and clinical programs 
respectively) and an administrative officer. Consultation with other staff occurred regularly via 
formal presentations as well as through the School’s curriculum management committee. 

What is curriculum mapping and why is it necessary? 

Curriculum is a complex mix of learning outcomes, content, the scope and sequence of objectives, 
teaching, learning and assessment methods, resources ... and more (Harden, 2001). Curriculum 
maps attempt to give order to these elements as well as allow the relationships between them to be 
ascertained, queried and maintained. Traditionally, much of this information is ‘owned’ or 
managed by academics in isolation from one another, even if they teach units in the same course. 
While unit coordinators generally work hard at ensuring their units are coherent and well- 
structured, they rarely come together to discuss how their units work together to achieve course 
learning outcomes. Consequently, it is possible for courses to address similar outcomes multiple 
times across a number of units or, worse still, for outcomes to not be addressed at all. Holycross 
wrote. 

An aspect of curricular development that has been relatively neglected is how curriculum developers 
communicate about the content of curriculum and track that content. How do faculty know what has 
been covered in the curriculum and when it has been addressed? (2006, p. 61). 

Curriculum mapping aims to address this question by making transparent to staff what is taught, 
how it is taught, when it is taught, the scope and sequence across the lifespan of a course, and any 
problems associated with these elements. When curriculum for a course is brought together in a 
central repository and presented as a whole, it is possible to spot gaps and overlaps thus providing 
a platform for all staff responsible for developing and delivering it to collectively plan and make 
decisions about how a more seamless outcome can be achieved. Furthermore, in bringing together 
the different elements of a course’s curriculum, the relationships and connections between these 
elements can be examined to discern how and if broad (institutional and / or professional) 
educational requirements are being fulfilled. 

A central point, or focus, however needs to be established around which the relationships can be 
built. The central point in the SoM’s curriculum map is the MBBS learning outcomes. Drawing on 
the concept of ‘constructive alignment’, it was decided that the curriculum map should provide a 
statement of intention as to what students were expected to learn at varying stages of the course. In 
keeping with the premise of constructive alignment (Biggs & Tang, 2007) that learning outcomes 
determine the design of learning activities and assessment tasks, mapping demonstrates alignment 
(or lack thereof) between the intended learning outcomes, how they are resourced (i.e., the 
learning activity assigned to address them), and how they are assessed. Importantly, anomalies 
associated with misalignment can be explored and fixed. 

Figure 1 illustrates the relationship between the specific learning objectives in the MBBS, the 
learning resources and the assessment program. However, as there are literally thousands of 
specific learning objectives across the four years of the MBBS, it was recognised that these 
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detailed, daily objectives, needed to be categorised and organised further. The process of arranging 
these objectives into modules, grouping them according to a learning foci and then linking them to 
broader unit goals eventually led to the development of the framework for the database structure. 



Figure 1: Constructive alignment informs the logic of the curriculum framework 

What is the logic underpinning our curriculum? 

Managing and ordering the thousands specific learning objectives was largely a bottom-up process 
of determining what larger course, institutional and professional expectations inform them. A 
university’s graduate attributes are an example of the institutional qualities students are expected 
to develop irrespective of the course they complete and they should be evident in the learning and 
assessment activities embedded in the course’s instructional design. For professional courses such 
as medicine, there are also the accrediting body’s standards to consider. This bottom-up process 
was at times hampered by an inability to link upwards due to incomplete data in the levels 
bridging the specific and broad outcomes. Top-down exploration of links often resolved some of 
these issues and was a useful quality assurance process for checking relationships as they 
unfolded. This iterative method of ordering and structuring the various levels of outcomes was 
time consuming but crucial as it led to the development of the School’s curriculum framework. 

This framework is an 'outcomes' based model where the statements in each level identify what 
students will have achieved at the end of a period of time (e.g., at the end of the MBBS course; at 
the end of a unit, at the end of a module, etc.). It is also hierarchical (see Figure 2) in that the top 
level outcome statements influence the outcome statements below it. The top level statements are 
also broad, overarching and fewer in number and are referred to as the macro learning outcomes. 
In contrast, the specific learning outcomes at level five relate to daily learning expectations and, 
therefore, are more detailed and descriptive. They are referred to as the micro learning outcomes, 
of which there are thousands across the four years of the course. 

Due to the relational nature of the curriculum framework, it is possible to explore the relationship 
between any one of the specific learning objectives and its relevant level four, three, two and one 
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outcome statements. Consequently, if the AMC were to ask the SoM today about how and when 
an AMC student outcome statement is taught in the course, Prudentia could swiftly show the 
related ‘children’ outcomes for all AMC student outcome statements right down to the specific 
learning outcomes, their resources and assessments. 



Figure 2: The School of Medicine’s curriculum framework 

A definition of each level of the SoM curriculum framework is provided below. 

Level la - AMC Student Outcome Statements are the knowledge, skills and attitudes students in 
Australian and New Zealand medical schools are expected to attain upon graduating from their 
MBBS course. 

Level lb - Notre Dame Graduate Attributes are the qualities, traits and skills students who 
complete courses at Notre Dame are expected to develop during their time at the University. 

Level 2 - MBBS Course Outcomes are the knowledge, skills and attitudes students are expected to 
achieve upon completion of the course. These statements are derived from the AMC student 
outcome statements and the Notre Dame graduate attributes. 

Level 3 - Unit Goals are the knowledge, skills and attitudes students are expected to achieve at the 
end of each unit. As the units in the MBBS at Notre Dame are year-long, these goals also define 
learning expectations that students must achieve before they are deemed ready to progress to the 
next year/unit. The Unit Goals are derived from the MBBS course outcomes. 
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Level 4 - Broad Learning Outcomes. Otherwise known as block or rotation outcomes, these 
outcome statements relate to the knowledge, skills and attitudes students are expected to achieve at 
the end of each two to four week module (of which there are multiple throughout a unit). The 
block/rotation outcomes are derived from the unit goals. 

Level 5 - Specific Learning Objectives are the knowledge, skills and attitudes students are 
expected to achieve at the end of each learning encounter (e.g., lecture, lab, tutorial, bedside 
teaching session, etc.) that are provided within each block/rotation. 

The specific learning objectives are also tagged with metadata relevant to the structure of the 
curriculum within the School of Medicine at Notre Dame. For example, each one is tagged with a 
domain that relates to basic clinical sciences (BCS), communication and clinical practice (CCP), 
population and preventative health (PPH) or personal and professional development (PPD). BCS is 
intended to comprise 60% of the first year curriculum; CCP 20% and PPH and PPD 10% 
respectively. This domain representation alters as students progress through the course such that 
by the time they reach fourth year, 60% of their learning is CCP related and 20% BCS. PPH and 
PPD represent 10% of the curriculum in each of the four years. Analyses can be run in Prudentia 
to determine the extent to which the curriculum outcomes actually reflect this required domain 
representation. 

As mentioned previously, in developing the SoM’s curriculum framework it was discovered that 
data in some levels were missing. For example, levels three and four, while important components 
of the curriculum logic, did not have outcomes that were specified and explicit. Without unit 
outcomes (level three), it is difficult to understand the end goal and how each component part 
(level four) contributes towards the achievement of these goals. Jacobs and Johnson (2009) noted 
that it is not uncommon for curriculum structures to be ‘well-kept secrets’, but that when they are 
clearly articulated, the transparency of the framework allows staff and students to stay on course 
throughout a program. 

Conversations about the structural logic of the SoM’s curriculum, although time consuming, were 
possibly the most important step in the development of Prudentia. They occurred between various 
groups and at various levels in the School’s governance structure and revealed multiple views and 
inconsistencies in how faculty view the logic of the MBBS. They also revealed multiple 
perceptions of what curriculum actually is and what a curriculum map should and should not do. 
Consequently, they were valuable professional development opportunities as well as a means of 
progressing the curriculum mapping system. 

What curriculum data should be mapped? 

Constructive alignment is central to the conceptual structure of Prudentia. Learning outcomes 
inform the design of learning activities and assessment tasks, and curriculum mapping is the 
process of aligning these three curricular elements. Outcome statements, therefore, constitute the 
primary source of data in Prudentia. As such, a relational database underpins its design which 
allows multiple data sets to be cross-referenced, and rules around their relationships to be 
established. The multiple data sets in this instance are the five learning outcome levels of the 
curriculum framework. As a hierarchical structure, rules define how data within these outcome 
levels relate to one another. For example, to be deemed ‘expressed’ in the course, an AMC student 
outcome statement must be linked down through the lower levels of the framework to one or more 
specific learning objectives. If it does not have a related specific learning objective, then the AMC 


5 



Journal of University Teaching & Learning Practice, Vol. 12 [2015], Iss. 4, Art. 9 


student outcome statement will be flagged with a red hazard sign, signalling to staff that an 
anomaly exists that requires investigation. 

Mapping, therefore, occurs at an ‘institutional’ level - where micro specific learning objectives are 
linked back up through the curriculum framework to course outcomes and accreditation standards, 
as well as graduate attributes. It also occurs, however, at the ‘program’ level, where specific 
learning objectives are linked to learning activities and assessment tasks. Therefore, learning 
activities (referred to as ‘resources’ in Prudentia ) and assessments also constitute ‘data’. 

In addition to outcomes, activities and assessment, Harden (2002) outlined other curricular 
elements that can be incorporated into a map. For example, content, learning location, timetabling 
information, staff details and curriculum management processes can also be included. In this 
regard, the “...curriculum map provides a multi-dimensional view of the curriculum and allows 
the user to look at the curriculum with different lenses...” (p. 11-12). The SoM, however, decided 
not to expand Prudentia’ s scope to include these extra elements, given the role of the University’s 
learning management system in handling many of them. 

How does a curriculum mapping system differ from a learning 
management system and should these two systems interface? 

A learning management system (LMS) is a web-based application “that supports the development, 
delivery, assessment and administration of courses...” (Clayton et al, 2014). Students enrolled in a 
course delivered through the LMS can gain access via authenticated accounts which they must log 
into. Teachers can manage the release of this content, set assignments, manage grades and 
encourage online collaboration via the synchronous and asynchronous collaboration tools. 

A curriculum mapping system (CMS), however, is a structured repository of outcomes data that 
has been stored in such a way that the scope and sequence across the lifespan of a course can be 
displayed and tracked. Repetition, redundancies and gaps in this lifespan can be identified and 
modified accordingly. In this regard, staff and students can look forwards in a course to 
“appreciate the future applicability of what they are currently studying” (Kerslake & McKendree, 
2006). Alignment between outcomes data can also be tracked to ensure institutional and course 
goals as well as accreditation standards are being met. In this regard, the role of a CMS is different 
to an LMS. One is a mechanism for managing and delivering course content to students; the other 
provides a means by which the curriculum framework of a course can be displayed and queried. 

During development of the ‘resources’ component of Prudentia, there was discussion around 
whether this feature should link to the School’s LMS ( Blackboard ) which is used to deliver 
weekly content to students. The resource feature in Prudentia provides a description of the activity 
that students will encounter in an effort to address the specific learning objective. The question 
was posed as to whether linking this description to its related content (e.g., lecture slides, 
electronic articles, lab sheets, etc.) in Blackboard would add value to the CMS. Certainly, the SoM 
did not want to create a separate storage area for content in Prudentia as this would create 
problems associated with multiple versions of content. By linking directly to the LMS, any 
updates to the content would automatically be registered in Prudentia. Whilst this may be an 
option for future development, it has been postponed so that staff can become familiar with the 
scope, purpose and defining features of Prudentia as a CMS and so that the lines between it and 
the LMS are not blurred. 
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Discussion 

Addressing the questions outlined above has been instrumental in designing and developing a 
useful electronic curriculum mapping system that reflects the curriculum framework of the SoM. 
The outcome, Prudentia, has the potential to transform how staff understand and interact with 
curriculum as well as improve curriculum review and management practices. However, this is 
dependent on staff using the application and consciously adopting it over other well-established 
practices. Despite its potential, implementation and training must be managed carefully to ensure it 
is successfully embedded into School systems and that it doesn’t become an expensive white 
elephant. 

In a study conducted into staff perceptions of curriculum mapping in a secondary school context. 
Shilling (2013) found that staff buy-in and commitment is critical to successful implementation. 
She found that those teachers who did not see the relevance of curriculum mapping to their 
specific planning needs were less likely to commit to it. This generally stemmed from a lack of 
communication and detailed explanation from school leaders that curriculum mapping does not 
equate to a loss of individuality in curriculum design and instruction. 

As such, roll-out of Prudentia has been guided by an implementation plan that addresses the 
technical up-skilling of staff in the context of meaningful and customised searches. Rather than 
run generic workshops for large groups, sessions have been held with one to three staff at a time so 
that their specific curricular needs can be used to focus navigation and exploration. The training 
session is guided by questions staff have about their area of curriculum and their role in either 
developing or delivering it rather than a generic one-size-fits all approach. Furthermore, training 
has been delivered to all staff, general and academic, at all levels and is part of the induction pack 
for new appointees. 

Importantly, curriculum management processes have changed with the introduction of Prudentia. 
No longer are changes to any aspect of the curriculum unilateral decisions made by individuals in 
isolation from others who teach into the MBBS. Instead, changes must be discussed at the 
operational level then referred to the curriculum management committee for overall agreement and 
ratification. Once this happens an administrator makes the appropriate changes in Prudentia. This 
process ensures that data in Prudentia constitutes the ‘master’ copy of curriculum outcomes from 
which all curriculum documents are developed. This addresses the problem of multiple (and often 
varying) print-based versions of curriculum being stored in multiple locations with little surety as 
to which version is the most up-to-date. As Kerslake and McKendree (2006) pointed out “... a 
curriculum map is not simply a one-off end product - it is a commitment to an ongoing process of 
data management ... The electronic map should not lag behind some other printed source”. 

Staff reactions to Prudentia and the new processes have been mixed, but mostly positive. Some 
individuals were reluctant due to the perception that it represented a doubling up of processes (i.e., 
that they would have to maintain their own curriculum records and Prudentia data). However, 
once there was recognition that Prudentia is the curriculum, as well as the tool to manage it, this 
reluctance is slowly dissipating. The user-friendly interface has also reduced initial anxieties about 
how Prudentia will impact daily practices. 

Furthermore, the red ‘hazards’ that flag anomalies in the data were at first alarming to staff, but 
they have, in many cases, become the impetus to clean data by either re-writing outcome 
statements or deleting them if redundant. Trainers have been eager to point out that curriculum in 
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complex courses such as medicine will rarely be perfect and that staff should not be intimidated by 
the red hazard icons. They are there simply to identify areas that need investigation. 

Evaluation of Prudentia has been informal to date, and based on anecdotal evidence collected 
during training sessions. A formal evaluation is planned for 2015 to measure staff attitudes and 
satisfaction levels. It will also assess behavioural changes at the individual, team (including cross¬ 
team collaboration), and curriculum governance levels. Importantly, evaluation and monitoring 
will be cyclical and an on-going endeavour to ensure timely identification of and response to staff 
issues as they arise. As Shilling pointed out, findings from evaluations should not only identify 
problems associated with curriculum mapping, but also highlight “and celebrate even modest 
successes to signify the importance of the initiative and promote its success and sustainability” (p. 
32). 

Prudentia’s capabilities 

Once clarity around the purpose of a curriculum map was established, the specification for what 
the SoM’s electronic CMS should do were outlined. The primary features were identified as 
follows. Prudentia should: 

1. provide a centralized, dynamic repository for the SoM’s curriculum data 

2. allow investigation of constructive alignment between intended learning outcomes, 
resources and assessments 

3. provide remote access for the large number of staff who work off-campus (and students 
eventually) 

4. have user-friendly browse and search capabilities 

5. allow for administration of the data to be managed internally within the SoM 

6. highlight anomalies in the relationships between data in the curriculum framework 

7. integrate with a medical language thesaurus to allow for synonyms of terms to be searched 

8. provide reporting and data export capabilities 

9. provide data analysis capabilities 

10 link all level five specific learning objectives to an assessment item bank 

These specifications can be classified as ‘second generation’ curriculum map capabilities, where 
the aim is to “support students and teachers in the learning and teaching process while also 
supporting administrative processes” (Watson et al, 2007, p. 357). First generation CMSs tended 
to focus primarily on supporting curriculum management and administration processes only. 

While Prudentia may have been conceived initially to satisfy an accreditation requirement, the 
development team also saw it as an opportunity to change how staff perceived and interacted with 
curriculum. To address the entrenched solo curriculum development and management practices, 
the holistic and integrated design of Prudentia encourages staff to view their ‘part’ of the 
curriculum as one piece in a larger jigsaw, and that in an effort for their piece to fit into place, staff 
must have knowledge of the shape and nature of other pieces around it. For this to happen, they 
must converse with colleagues about how multiple parts of the curriculum collectively contribute 
to students reaching outcomes. In short, Prudentia encourages and supports collaborative data 
interrogation and decision making. 

For the first time since the inception of the course (2005), staff are able to view and examine the 
curriculum as a whole. They can foreground specific topics and/or domains and explore intended 
outcomes across all four years to see if there are gaps or overlaps. And, of course, they can 
examine the relationships these intended outcomes have up and down the hierarchical curriculum 
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framework. Anecdotal evidence suggests that, even the most reluctant staff are pleased with this 
development. 

Another powerful feature of Prudentia is its interface with the Unified Medical Language System 
(UMLS), a metathesaurus comprising biomedical and health-related terms from over 100 
controlled medical language sources (e.g., SNOMED, MeSH, WHO, ICD-10, etc). Prudentia’s 
search function searches the thousands of specific learning objectives (level five of the curriculum 
framework). However, given the many ways in which medical terms can be expressed, an exact 
match keyword search function would not retrieve a full set of results. By linking to the UMLS, 
which is a free service that is updated quarterly, synonyms of key words are also searched thus 
increasing the amount of SLOs that are retrieved. 

To date, the first seven specifications (see list above) have been achieved. If work on Prudentia 
was to cease today, it would still be considered a valuable tool based on the centralised repository 
feature alone. However, work continues and the assessment database, as well as data reporting / 
export capabilities, is planned for 2015. A ‘student view’ is also planned for 2015 in the interests 
of ensuring transparency around curricular goals and to further encourage students to take control 
of their own learning. 

Conclusion 

Prudentia was initially conceived of as a solution to satisfy the AMC’s request to demonstrate 
how standards were being taught in the MBBS curriculum. However, its value to the School is 
much more than this, to the extent that it is slowly transforming how staff perceive and manage 
curriculum. Rather than view curriculum simply as a lecture on XYZ, or as a tutorial about a 
specific patient case, staff are beginning to recongise the importance of intended learning 
outcomes as something that drives the learning / teaching experience. In a course such as medicine 
where high stakes assessments tend to dominate the curriculum, the constructive alignment 
structure of Prudentia is re-focusing the power of the intended learning outcome as a starting point 
for successful assessment outcomes. 

A training and implementation plan has been carefully designed and rolled out to ensure that 
Prudentia is embedded into School processes and staff practices such that it doesn’t become a 
white elephant. Up-take is increasing and staff can mostly see the benefits and value in Prudentia. 
It is important to note, however, that some staff will want to maintain traditional approaches to 
curriculum development and management. An evaluation strategy will be developed in 2015 to 
capture the extent to which staff have incorporated it into their practices. 

Prudentia is a user-friendly application that reflects the SoM’s curriculum framework. Whilst the 
development of the curriculum framework was slow, the application would not be the success that 
it is today without this work. Any other School considering developing their own electronic 
curriculum mapping system would be wise to complete this task first before building the 
technology. 
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